Home | | Latest | About | Random
# 15 Span of a set of vectors.
Let us go further with the idea of linear combinations. Sometimes we wish to discuss the totality of linear combinations -- that is, all the linear combinations of some set of objects. This is the idea of **span**.
> **Definition. Span.**
> Let $v_{1},v_{k},\ldots,v_{k}$ be some objects where taking linear combinations of them make sense. Then we denote $$
\operatorname{span} (v_{1}, v_{2},\ldots, v_{k})=\{c_{1}v_{1}+c_{2}v_{2}+\cdots+c_{k}v_{k}:c_{k} \text{ scalars}\}
$$to be the **set of all linear combinations** of the objects $v_{1},v_{2},\ldots,v_{k}$, over some choice of scalars.
**Remark.** If we don't mention the scalars, then we have to infer them from context. But sometimes we will specify what the choice of scalars are. For example, $\operatorname{span}_{\mathbb{R}}(v_{1},v_{2})$ is the set of all linear combinations of the form $c_{1}v_{1}+c_{2}v_{2}$ where $c_{1},c_{2}$ is any real number, while $\operatorname{span}_{\mathbb{C}}(v_{1},v_{2})$ is the set of all linear combinations of the form $c_{1}v_{1}+c_{2}v_{2}$ where $c_{1},c_{2}$ is any complex number.
**Remark.** The $\operatorname{span}(v_{1},\ldots,v_{k})$ is a set of things. So we get a collection, and it can be infinite. It is all the linear combination we can make using the objects ("vectors") $v_{1},\ldots,v_{k}$.
**Remark.** Generally it causes no confusion to write $\operatorname{span}\{v_{1},\ldots,v_{k}\}$ or $\operatorname{span}(v_{1},\ldots,v_{k})$.
**Remark.** This notation of span is useful to describe a collection of vectors or objects all at once. Let us see some examples and some geometric descriptions of them when we can.
---
**Example.** What is $\operatorname{span}\{\begin{bmatrix} 1\\2\end{bmatrix}\}$? Assuming scalars are in reals $\mathbb{R}$.
Well, this is the **set** of vectors of the form $c\begin{bmatrix}1\\2\end{bmatrix}=\begin{bmatrix} c \\2c\end{bmatrix}$, for all $c\in\mathbb{R}$. That is, $$
\operatorname{span} \{\begin{bmatrix}
1 \\
2
\end{bmatrix}\} = \{c\begin{bmatrix}
1\\2
\end{bmatrix}:c\in\mathbb{R}\}.
$$So this is a lot of vectors (infinitely many of them)! Every scalar multiple of $\begin{bmatrix}1\\2\end{bmatrix}$. We can try to understand this geometrically. The set of all vectors in $\operatorname{span}\{\begin{bmatrix} 1\\2\end{bmatrix}\}$ forms a line in the plane $\mathbb{R^{2}}$, which is shown as the $\color{blue}\text{blue line}$ below:
```tikz
\begin{document}
\begin{tikzpicture}
\tikzset{>=latex}
% Axes
\draw[gray] (-2,0) -- (2,0);
\draw[gray] (0,-2) -- (0,2);
\node[gray] at (1.7,-1.7) {\(\mathbf{R}^2\)};
% Blue line
\draw[blue, thick] (-1,-2) -- (1,2);
\node[blue] at (1.7, 1.5) {span\(\{{1 \choose 2}\}\)};
% Black vector
\draw[black, very thick, ->] (0,0) -- (0.5,1);
\node at (0.6,0.6) {\(1 \choose{2}\)};
\end{tikzpicture}
\end{document}
```
Let us continue to think about span of vectors from the Euclidean space $\mathbb{R^{n}}$, when $n\le 3$ so we can visualize.
---
**Example.** What is $\operatorname{span}(\begin{bmatrix}0\\0\\0\end{bmatrix})$? Assuming scalars in reals.
Let us think about every linear combination you can get from $\begin{bmatrix}0\\0\\0\end{bmatrix}$. Well you can only get one vector out of, which is $\begin{bmatrix}0\\0\\0\end{bmatrix}$ itself! So, $\operatorname{span}(\begin{bmatrix}0\\0\\0\end{bmatrix})$ is a set of vectors with only one vector in it, namely $\begin{bmatrix}0\\0\\0\\\end{bmatrix}$. That is $$
\operatorname{span}(\begin{bmatrix}0\\0\\0\end{bmatrix}) =\{\begin{bmatrix}0\\0\\0\\\end{bmatrix}\}.
$$A set with only one and exactly one thing in it is sometimes called a **singleton set**. Geometrically, this describes a single point in Euclidean 3-space:
```tikz
\begin{document}
\begin{tikzpicture}[x={(-0.8cm,-0.8cm)}, y={(1cm,0cm)}, z={(0cm,1cm)}]
% Axes
\draw[gray,->] (-2,0,0) -- (2,0,0) node[anchor=north east]{\(x\)};
\draw[gray,->] (0,-2,0) -- (0,2,0) node[anchor=north west]{\(y\)};
\draw[gray,->] (0,0,-2) -- (0,0,2) node[anchor=south]{\(z\)};
% Point at origin
\fill[blue] (0,0,0) circle (2pt);
\end{tikzpicture}
\end{document}
```
---
**Example.** What is $\operatorname{span}(\begin{bmatrix}1\\2\\1\end{bmatrix},\begin{bmatrix}1\\2\\2\end{bmatrix})$? Assuming scalars in reals.
Well, this is the set $$
\operatorname{span}(\begin{bmatrix}1\\2\\1\end{bmatrix},\begin{bmatrix}1\\2\\2\end{bmatrix}) = \{c_{1}\begin{bmatrix}1\\2\\1\end{bmatrix} + c_{2}\begin{bmatrix}1\\2\\2\end{bmatrix}:c_{1},c_{2}\in \mathbb{R}\}.
$$Geometrically, we see that the vectors $v_{1} = \begin{bmatrix}1\\2\\1\end{bmatrix}$ and $v_{2}=\begin{bmatrix}1\\2\\2\end{bmatrix}$ are pointing in different directions. If we imagine all the linear combinations they can make, their span forms a **plane** in $\mathbb{R}^{3}$:
```tikz
\begin{document}
\begin{tikzpicture}[x={(-0.8cm,-0.6cm)}, y={(1cm,0cm)}, z={(0cm,1cm)}]
\tikzset{>=latex}
% Axes
\draw[gray,->] (-2,0,0) -- (2,0,0) node[anchor=north east]{\(x\)};
\draw[gray,->] (0,-2,0) -- (0,2,0) node[anchor=north west]{\(y\)};
\draw[gray,->] (0,0,-2) -- (0,0,2) node[anchor=south]{\(z\)};
% two vectors
\filldraw[blue!30, opacity=0.4] (1.5,3,1.5) -- (1.5,3,3) -- (-1.5,-3,-1.5) -- (-1.5,-3,-3) -- cycle;
\draw[black,thick,->] (0,0,0) -- (1,2,1);
\node[anchor=west] at (1.2,2.2,1.2) {\(v_1\)};
\draw[black,thick,->] (0,0,0) -- (1,2,2);
\node[anchor=west] at (1.2,2.2,2.2) {\(v_2\)};
\node[blue, anchor=west] at (3,0,0) {span\((v_1,v_2)\)};
\end{tikzpicture}
\end{document}
```
---
**Example.** Of course, the span of two vectors need not always be a plane. Think about what is $\operatorname{span}(\begin{bmatrix}1\\2\\1\end{bmatrix},\begin{bmatrix}2\\4\\2\end{bmatrix})$?
Notice that the two vectors are already scalar multiples of each other. So $\operatorname{span}(\begin{bmatrix}1\\2\\1\end{bmatrix},\begin{bmatrix}2\\4\\2\end{bmatrix})$ really is a line in $\mathbb{R}^{3}$! Here is a diagram, if we write $\vec v_{1}=\begin{bmatrix}1\\2\\1\end{bmatrix},\vec v_{2}=\begin{bmatrix}1\\2\\1\end{bmatrix}$, the $\color{blue}\text{blue}$ line is the span of these two vectors:
```tikz
\begin{document}
\begin{tikzpicture}[x={(-0.8cm,-0.6cm)}, y={(1cm,0cm)}, z={(0cm,1cm)}]
\tikzset{>=latex}
% Axes
\draw[gray,->] (-2,0,0) -- (2,0,0) node[anchor=north east]{\(x\)};
\draw[gray,->] (0,-2,0) -- (0,2,0) node[anchor=north west]{\(y\)};
\draw[gray,->] (0,0,-2) -- (0,0,2) node[anchor=south]{\(z\)};
% two vectors
\draw[blue] (-2.5,-5,-2.5)--(2.5,5,2.5);
\node[blue, anchor=north] at (3,0,1.7) {span\((v_1,v_2)\)};
\draw[black,very thick,->] (0,0,0) -- (1,2,1);
\node[anchor=north] at (1,2,1) {\(v_1\)};
\draw[black,very thick,->] (0,0,0) -- (2,4,2);
\node[anchor=north] at (2,4,2) {\(v_2\)};
\end{tikzpicture}
\end{document}
```
In this case, we can "throw away redundancy", and rewrite this span as the following: $$
\operatorname{span}(\begin{bmatrix}1\\2\\1\end{bmatrix},\begin{bmatrix}2\\4\\2\end{bmatrix})=\operatorname{span}(\begin{bmatrix}1\\2\\1\end{bmatrix}).
$$Think about why this is really the case!
---
Later we will discuss how to remove these "redundancies" in a span expression in general.
Also, later we will investigate when and why we get a plane sometimes, or a line (as well as higher "dimensional") generalizations.
---
Note, we now can ask a new kind of question:
> Is a given vector $\vec b$ in the span of a set of vectors $v_{1},v_{2},\ldots,v_{k}$. That is, is $\vec b \in \operatorname{span}(v_{1},v_{2},\ldots,v_{k})$?
Let us think about what this is asking. To ask if $\vec b$ is in the $\operatorname{span}(v_{1},v_{2},\ldots,v_{k})$ is to ask if $\vec b$ is a linear combination of $v_{1},v_{2},\ldots,v_{k}$ (since $\operatorname{span}(v_{1},v_{2},\ldots,v_{k})$ is all such linear combinations!). But to ask if $\vec b$ is a linear combination of $v_{1},v_{2},\ldots,v_{k}$, it is to ask whether we can solve the equation $\vec b = c_{1}\vec v_{1}+c_{2}\vec v_{2}+\cdots c_{k}\vec v_{k}$ for some scalars $c_{1},c_{2},\ldots,c_{k}$. But this is the same as asking us to solve the matrix-vector equation (linear system) $$
\begin{bmatrix}
\vec v_{1} & \vec v_{2} & \cdots & \vec v_{k}
\end{bmatrix} \begin{bmatrix}
c_{1} \\ c_{2} \\ \vdots \\ c_{k}
\end{bmatrix} = \vec b
$$So if we write matrix $A$ as the matrix whose columns are $\vec v_{1},\vec v_{2},\ldots,v_{k}$, then we are asking if we can solve the linear system $A\vec c = \vec b$, for some weights vector $\vec c = \begin{bmatrix}c_{1} \\ c_{2} \\ \vdots \\ c_{k}\end{bmatrix}$. And we know how to solve these: We row reduce the augmented matrix $\begin{bmatrix}A & \vdots & \vec b\end{bmatrix}$ to an echelon form!
**Example.** Is the vector $\begin{bmatrix}1\\2\\3\end{bmatrix}$ in $\operatorname{span}(\begin{bmatrix}1\\2\\1\end{bmatrix},\begin{bmatrix}1\\2\\2\end{bmatrix})$? Prove it with calculation. If so, find the explicit linear combination of $\begin{bmatrix}1\\2\\1\end{bmatrix},\begin{bmatrix}1\\2\\2\end{bmatrix}$ that makes $\begin{bmatrix}1\\2\\3\end{bmatrix}$.
Using what we understand above, this boils down to setting up the augmented matrix $$
\begin{bmatrix}
1 & 1 & \vdots & 1 \\
1 & 2 & \vdots & 2 \\
1 & 2 & \vdots & 3
\end{bmatrix} \sim \begin{bmatrix}
1 & 1 & \vdots & 1 \\
0 & 1 & \vdots & 1 \\
0 & 1 & \vdots & 2
\end{bmatrix} \sim \begin{bmatrix}
1 & 1 & \vdots & 1 \\
0 & 1 & \vdots & 1 \\
0 & 0 & \vdots & 1
\end{bmatrix}
$$which we see is **inconsistent**, hence we conclude that the vector $\begin{bmatrix}1\\2\\3\end{bmatrix}$ is **not** in the span of $\operatorname{span}(\begin{bmatrix}1\\2\\1\end{bmatrix},\begin{bmatrix}1\\2\\2\end{bmatrix})$ !
Now geometrically what does this mean? Earlier we reasoned that $\operatorname{span}(\begin{bmatrix}1\\2\\1\end{bmatrix},\begin{bmatrix}1\\2\\2\end{bmatrix})$ is some plane in $\mathbb{R}^{3}$. So for the vector $\color{red} \vec b =\begin{bmatrix}1\\2\\3\end{bmatrix}$ to be **not in this span** means it is "sticking out of the plane", as shown below!
```tikz
\begin{document}
\begin{tikzpicture}[x={(-0.8cm,-0.6cm)}, y={(1cm,0cm)}, z={(0cm,1cm)}]
% Axes
\draw[gray,->] (-2,0,0) -- (2,0,0) node[anchor=north east]{\(x\)};
\draw[gray,->] (0,-2,0) -- (0,2,0) node[anchor=north west]{\(y\)};
\draw[gray,->] (0,0,-2) -- (0,0,2) node[anchor=south]{\(z\)};
% two vectors
\filldraw[blue!30, opacity=0.4] (1.5,3,1.5) -- (1.5,3,3) -- (-1.5,-3,-1.5) -- (-1.5,-3,-3) -- cycle;
\draw[black,thick,->] (0,0,0) -- (1,2,1);
\node[anchor=west] at (1.2,2.2,1.2) {\(v_1\)};
\draw[black,thick,->] (0,0,0) -- (1,2,2);
\node[anchor=west] at (1.2,2.2,2.2) {\(v_2\)};
\node[blue, anchor=west] at (3,0,0) {span\((v_1,v_2)\)};
\draw[red,thick,->] (0,0,0) -- (1,2,3);
\node[red, anchor=west] at (1,2,3) {\(\vec b\)};
\end{tikzpicture}
\end{document}
```
**Example.** What is $\operatorname{span}\{\begin{bmatrix}1\\0\\0\end{bmatrix},\begin{bmatrix}0\\1\\0\end{bmatrix},\begin{bmatrix}0\\0\\1\end{bmatrix}\}$? Assume the scalars are reals $\mathbb{R}$.
Note this is the set of all possible linear combinations of the three vectors $\begin{bmatrix}1\\0\\0\end{bmatrix},\begin{bmatrix}0\\1\\0\end{bmatrix},\begin{bmatrix}0\\0\\1\end{bmatrix}$, which has the form $$
c_{1}\begin{bmatrix}1\\0\\0\end{bmatrix}+c_{2}\begin{bmatrix}0\\1\\0\end{bmatrix}+c_{3}\begin{bmatrix}0\\0\\1\end{bmatrix} = \begin{bmatrix}c_{1}\\c_{2}\\c_{3}\end{bmatrix},
$$where $c_{1},c_{2},c_{3}$ are free. If we let $c_{1},c_{2},c_{3}$ to be any real numbers, then we just get all the vectors in $\mathbb{R}^{3}$! Hence $$\operatorname{span}\{\begin{bmatrix}1\\0\\0\end{bmatrix},\begin{bmatrix}0\\1\\0\end{bmatrix},\begin{bmatrix}0\\0\\1\end{bmatrix}\}=\mathbb{R}^{3}.$$
**Notational note.** We have been writing $\mathbb{R^{2}}$, $\mathbb{R^{3}}$, etc. For us (for now), we will denote $\mathbb{R}^{n}$ to be $$
\mathbb{R}^{n}= \{\begin{bmatrix}
c_{1} \\ c_{2} \\ \vdots \\c_{n}
\end{bmatrix}:c_{1},c_{2},\ldots,c_{n}\in \mathbb{R}\},
$$and we sometimes call $\mathbb{R}^{n}$ the **Euclidean $n$-space**.
**Some technical note here:** More abstractly, $\mathbb{R}^{n}$ is the set of all $n$-tuples whose coefficients are in $\mathbb{R}$, so $\mathbb{R}^{n}=\{(c_{1},c_{2},\ldots,c_{n}):\forall k\ c_{k}\in \mathbb{R}\}$. Here our choice of "all the column vectors of length $n$" to represent $\mathbb{R}^{n}$ is not quite technically "correct" but not too far off -- it is **isomorphic** to it. But if we squint we can ignore this distinction for now. Later if we really care about the distinction of "column vectors" ($n\times 1$ matrices) or even "row vectors" ($1\times n$ matrices), we will have a more specific notation for them.
As mentioned in the beginning, the idea of **span can be used to describe many objects all at once** (usually related to "all linear combinations of a set of things". as that is the definition of span).
**Example.** Let us consider the collection of all real polynomials of the form $ax+b$, where $a,b$ are some real numbers. Do you see how we can describe all such real polynomials $\{ax+b:a,b\in \mathbb{R}\}$? That's right, we can write it as $\operatorname{span}(x,1)$, or $\operatorname{span}_{\mathbb{R}}(1,x)$ if we want to emphasize the weights are all reals.
**Example.** The homogeneous differential equation $y''-y=0$ turns out to have general solutions of the form $y=c_{1}e^{x}+c_{2}e^{-x}$ for any $c_{1},c_{2}$ scalars. We can describe this set of solutions as $\operatorname{span}(e^{x},e^{-x})$.
**Example.** The set of all $2\times 2$ matrices over the reals, namely $\{\begin{bmatrix}a & b \\ c & d\end{bmatrix}:a,b,c,d\in\mathbb{R}\}$ can be written as $\operatorname{span}_{\mathbb{R}}\{\begin{bmatrix}1 & 0 \\ 0 & 0\end{bmatrix}, \begin{bmatrix}0 & 1 \\ 0 & 0\end{bmatrix}, \begin{bmatrix}0 & 0 \\ 1 & 0\end{bmatrix}, \begin{bmatrix}0 & 0 \\ 0 & 1\end{bmatrix}\}$.